Field of the Invention
The present invention relates to systems and methods used for providing real-time visual feedback to automatically control actions of multiple autonomous robots, and more particularly to those systems and methods that control actions of nano-robots with nano-manipulators.
Description of the Related Art
Numerous conventional solutions found in the art provide systems and methods of controlling actions of robots to enable manipulation tasks. Among such conventional solutions lie the U.S. Pat. No. 6,967,335, the U.S. Pat. No. 7,297,946, U.S. Pat. No. 6,891,170, U.S. Pat. No. 6,862,924, the European project titled “Smart and small robots for the micro-world”, the scientific article titled “Microrobot system for automatic nanohandling inside a scanning electron microscope” by Fatikow, S., et al published on May, 2006, and the scientific article titled “Microrobot system for automatic nanohandling inside a scanning electron microscope” by Fatikow, S., et al published on June, 2007.
U.S. Pat. No. 6,967,335 discloses a system and method to enable manipulation of a sample under study with a microscope. In one embodiment, a manipulation system is adaptable for interfacing with any of a plurality of different types of microscopes, such as a Transmission Electron Microscope (TEM) and a Scanning Electron Microscope (SEM), and further comprises at least one manipulation mechanism operable to manipulate a sample. In another embodiment, a manipulation system is capable of being detachably coupled to a microscope, such as a TEM, and comprises a plurality of manipulator mechanisms for manipulating a sample. In a preferred embodiment, the manipulation system comprises both an adjustable interface such that it is capable of selectively coupling with any of a plurality of different microscope interfaces and a plurality of manipulator mechanisms integrated therein that are controllably operable for manipulating a sample.
U.S. Pat. No. 7,297,946 provides an automated nano-manipulation system for manufacturing a nano-scale structure. The system includes: a design model for the nano-scale structure; image data of a sample surface upon which the nano-scale structure is to be manufactured; a movable member configured to perform a nano-manipulation operation on the sample surface; and a path planning subsystem adapted to receive the design model and the image data. The path planning subsystem generates path data indicative of a path for traversing the movable member along the sample surface such that the movable member manipulates one or more randomly distributed nano-objects in accordance with the design model.
In U.S. Pat. No. 6,891,170, a modular manipulation system and method for using such modular manipulation system for manipulating a sample under study with a microscope are provided. According to at least one embodiment, a platform is provided that comprises an interface to a microscope, a sample stage, and a plurality of interfaces for receiving manipulator modules for manipulating a sample arranged on the sample stage. Preferably, the plurality of interfaces for receiving manipulator modules are each capable of detachably coupling a manipulator module to the platform. Thus, in a preferred embodiment, a user may selectively couple one or more desired manipulator modules to the platform to enable a desired type of manipulation to be performed on a sample under study. Accordingly, a preferred embodiment enables great flexibility in configuring a manipulation system in a desired manner.
U.S. Pat. No. 6,862,924 provides a method for performing nano-manipulations using an atomic force microscope. The method includes: performing a nano-manipulation operation on a sample surface using an atomic force microscope; determining force data for forces that are being applied to the tip of the cantilever during the nano-manipulation operation, where the force data is derived along at least two perpendicularly arranged axes; and updating a model which represents the topography of the sample surface using the force data.
In the European project titled “Smart and small robots for the micro-world”, there is provided a nano-handling station operating in an SEM. Two robot platforms for coarse positioning (semiautonomous and a fixed) were developed in this project, wherein the design of the robots allow an easy conversion of the desktop station for different applications. Beside the SEM, micro video cameras and tactile-/force microsensors have been developed and integrated. Under such project, real-time image recognition and real-time processing of acquired force information were adopted.
Fatikow, S., et al., (2006) developed an automated nano-handling cell in a scanning electron microscope (SEM) is presented. An experimental setup is shown, in which two micro robots cooperate in the vacuum chamber of an SEM. A client-server control system that can integrate various micro robots and sensors has been developed and evaluated by automatic handling of TEM (transmission electron microscope) lamellae. The robots are controlled in a closed-loop way by using images from several CCD cameras and from the SEM. Algorithms for real-time processing of noisy SEM images have been implemented and tested. The hardware setup of the nano-handling cell consists of an SEM, two multi-DOF micro robots and the sensors for pose and contact detection.
Fatikow, S., et al., (2007) developed an automated nano-handling station in an SEM. In a described experimental setup, two mobile micro robots cooperate in the vacuum chamber of the SEM. The robots are positioned by a closed-loop controller with sensor data, which is provided by three charge-coupled device cameras and the SEM. Continuous pose estimation is carried out by processing noisy SEM images in real time. To enable the automation of complex tasks, a client-server control system that can integrate various micro robots and sensors is introduced. The overall system was evaluated by automatic handling of transmission electron microscope lamellae.